MATHEMATICAL REASONING BENCHMARKS

Think Outside the Policy: In-Context Steered Policy Optimization (2026.findings-acl)

Challenge:	TALMs have been successfully employed in question-answering benchmarks, but their efficacy on complex mathematical reasoning benchmarks are open research questions.
Approach:	They propose a tool-augmented large language model for mathematical reasoning that enhances the skillset of large language models (LLMs) by 13.5%.
Outcome:	The proposed model achieves better accuracy and better knowledge retrieval performance than existing tools.

Copied to clipboard

Hsiu-Yuan Huang, Chenming Tang, Weijie Liu, Clive Bai, Saiyong Yang, Yunfang Wu

Beyond Stochastic Exploration: What Makes Training Data Valuable for Agentic Search (2026.findings-acl)

Challenge:	Existing Reinforcement Learning from Verifiable Rewards (RLVR) methods exhibit limited exploration due to reliance on on-policy rollouts which are limited to the current policy’s distribution, resulting in narrow trajectory diversity.
Approach:	They propose a framework that leverages the in-context learning capability of Large Reasoning Models to provide expert guidance using existing datasets.
Outcome:	The proposed framework improves RLVR performance and training stability on mathematical reasoning benchmarks.

Copied to clipboard

Chuzhan Hao, Wenfeng Feng, Guochao Jiang, Guofeng Quan, Guohua Liu, Yuewei Zhang

AdaptFlow: Adaptive Workflow Optimization via Meta-Learning (2025.findings-emnlp)

Challenge:	Existing RL-based search agents rely on stochastic exploration, leading to inefficient reasoning trajectories and unstable training.
Approach:	They propose a framework to enhance the performance and training stability of search agents by transforming raw reasoning trajectories into hierarchical experience knowledge.
Outcome:	The proposed framework exhibits strong cross-task and cross-algorithm generalizations on multiple complex agentic search and mathematical reasoning benchmarks.

Copied to clipboard

Runchuan Zhu, Bowen Jiang, Lingrui Mei, Fangkai Yang, Lu Wang, Haoxiang Gao, Fengshuo Bai, Pu Zhao, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang

Learning Like Humans: Advancing LLM Reasoning Capabilities via Adaptive Difficulty Curriculum Learning and Expert-Guided Self-Reformulation (2025.emnlp-main)

Challenge:	Existing approaches to large language models rely on static templates or manual workflows.
Approach:	AdaptFlow is a language-based meta-learning framework inspired by model-agnostic meta- learning.
Outcome:	AdaptFlow outperforms manual and automated workflows on question answering, code generation and mathematical reasoning benchmarks.

Copied to clipboard

Enci Zhang, Xingang Yan, Wei Lin, Tianxiang. Zhang, Lu Qianchun

Supervised Optimism Correction: Be Confident When LLMs Are Sure (2025.findings-acl)

Challenge:	Extensive experiments on challenging mathematical reasoning benchmarks demonstrate that these human-inspired strategies synergistically and significantly enhance performance.
Approach:	They propose to use Adaptive Difficulty Curriculum Learning and Expert-Guided Self-Reformulation to improve model performance.
Outcome:	Extensive experiments on mathematical reasoning benchmarks show that the proposed strategies synergistically and significantly improve performance over the baseline model.

Copied to clipboard

Junjie Zhang, Rushuai Yang, Shunyu Liu, Ting-En Lin, Fei Huang, Yi Chen, Yongbin Li, Dacheng Tao

Self-Reflective Generation at Test Time (2026.acl-long)

Challenge:	Recent advances in Large Language Models (LLMs) have demonstrated remarkable success across diverse tasks such as instruction following, code generation, and medical diagnosis.
Approach:	They propose a supervised fine-tuning-based auxiliary loss for Q-value estimations during supervised refinement.
Outcome:	The proposed method outperforms beam search on GSM8K, MATH, and GAOKAO on reasoning benchmarks.

Copied to clipboard

Jian Mu, Qixin Zhang, Zhiyong Wang, Menglin Yang, Shuang Qiu, Chengwei Qin, Zhongxiang Dai, Yao Shu

Ensembling Large Language Models with Process Reward-Guided Tree Search for Better Complex Reasoning (2025.naacl-long)

Challenge:	Existing self-reflection mechanisms are reactive and inefficient for large language models . a fundamental tension persists between the ability to execute complex multi-step reasoning and the ability of the model to generate coherent traces.
Approach:	They propose a test-time framework that reflects before generating at uncertain points . SRGen utilizes dynamic entropy thresholding to identify high-uncertainty tokens .
Outcome:	The proposed framework can significantly strengthen large language models' reasoning process.

Copied to clipboard

Sungjin Park, Xiao Liu, Yeyun Gong, Edward Choi

Evolving Sparsity: Leveraging Token Importance Dynamics for Efficient LLM Decoding with Sparse Attention (2026.acl-long)

Challenge:	Existing methods for ensembling language models fail to address complex reasoning tasks.
Approach:	They propose a framework for process-level ensembling of large language models using Monte Carlo tree search.
Outcome:	The proposed framework outperforms both language model decoding and language model ensemble methods on five reasoning benchmarks.

Copied to clipboard

Ruizi Han, Miao Zhang, Ziyue Qiao, Liqiang Nie

Green & Sustainable NLP

Challenge:	Efficient long-context inference remains a major challenge for large language models (LLMs), as the cost of attention computation during auto-regressive decoding grows linearly with the context length.
Approach:	They propose to model token importance as a dynamic process that evolves over decoding steps and propagates through model layers.
Outcome:	The proposed method outperforms baseline sparse attention methods and achieves speedups of up to 5.36 for attention latency and 2.33 for end-to-end decoding.

Adapt to Thrive! Adaptive Power-Mean Policy Optimization for Improved LLM Reasoning (2026.findings-acl)

Copied to clipboard

Yiming Huang, Zhenbo Shi, Shuzheng Gao, Cuiyun Gao, Peiyi Han, Chuanyi Liu

Sticker-TTS: Learn to Utilize Historical Experience with a Sticker-driven Test-Time Scaling Framework (2025.emnlp-main)

Challenge:	Existing methods for reinforcement learning with verifiable rewards (RLVR) rely on static objective functions and rigid clipping strategies that misalign with the model’s evolving reasoning capabilities.
Approach:	They propose to incorporate Power-Mean Policy Optimization (PMPO) and Feedback-Adaptive Clipping (FAC) to overcome limitations of static mechanisms.
Outcome:	Extensive experiments on nine reasoning tasks show the proposed paradigm outperforms state-of-the-art methods.

Copied to clipboard

Jie Chen, Jinhao Jiang, Yingqian Min, Zican Dong, Shijie Wang, Xin Zhao, Ji-Rong Wen

O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning (2026.findings-acl)

Challenge:	Large reasoning models have exhibited strong performance on complex reasoning tasks, but current test-time scaling methods rely on redundant sampling and ignore historical experience utilization.
Approach:	They propose a test-time scaling framework that coordinates three collaborative LRMs to iteratively explore and refine solutions guided by historical attempts.
Outcome:	The proposed framework surpasses strong baselines on three mathematical reasoning benchmarks, including AIME-24, AIME-25, and OlymMATH.

Copied to clipboard

Haotian Luo, Haiying He, Yibo Wang, Shiwei Liu, Wei Li, Xiaochun Cao, Dacheng Tao, Naiqiang Tan, Li Shen

How to Allocate, How to Learn? Dynamic Rollout Allocation and Advantage Modulation for Policy Optimization (2026.findings-acl)

Challenge:	Recent long-thought reasoning models adopt extended reasoning processes similar to how humans ponder over complex problems.
Approach:	They propose a model that uses RL-style fine-tuning to reduce inference overhead while maintaining accuracy.
Outcome:	The proposed model reduces inference overhead while maintaining accuracy.

Copied to clipboard

Yangyi Fang, Jiaye Lin, Xiaoliang Fu, Cong Qin, Haolin Shi, Chaowen Hu, Lu Pan, Ke Zeng, Xunliang Cai

Guided by Gut: Efficient Test-Time Scaling with Reinforced Intrinsic Confidence (2026.acl-long)

Challenge:	Existing methods for reinforcement learning with verifiable rewards are limited by the complexity of the problem and the complexity.
Approach:	They propose a theoretically-grounded dual-pronged optimization framework for reinforcement learning with verifiable rewards that compensates for gradient attenuation of high-confidence correct actions while utilizing entropy changes as computable indicators to stabilize excessive update magnitudes.
Outcome:	The proposed framework compensates for gradient attenuation of high-confidence correct actions while utilizing entropy changes as computable indicators to stabilize excessive update magnitudes.

Copied to clipboard

Amirhosein Ghasemabadi, Keith G. Mills, Baochun Li, Di Niu

Understanding and Preventing Entropy Collapse in RLVR with On-Policy Entropy Flow Optimization (2026.findings-acl)

Challenge:	Guided by Gut (GG) is an efficient self-guided TTS framework for Large Language Models (LLMs) that performs step-by-step reasoning at a low cost without any reward models or verifiers.
Approach:	They propose a self-guided TTS framework that enables LLMs to perform step-by-step reasoning at a low cost without any reward models or verifiers.
Outcome:	Empirical evaluations show that GG performs better than TTS with PRMs while reducing GPU memory usage by up to 10.

Copied to clipboard

Huimin Xu, Shuai Zhao, Xiaobao Wu, Anh Tuan Luu

Mathador-LM: A Dynamic Benchmark for Mathematical Reasoning on Large Language Models (2024.emnlp-main)

Challenge:	Existing RLVR algorithms suffer from entropy collapse, leading to premature determinism and unstable optimization.
Approach:	They propose an adaptive entropy flow balancing mechanism that rescales entropic-increasing and enotro-decreazing updates according to their contributions to enthroy change.
Outcome:	The proposed method outperforms existing RLVR algorithms on six reasoning benchmarks.

Copied to clipboard

Eldar Kurtic, Amir Moeini, Dan Alistarh

Unlocking Exploration in RLVR: Uncertainty-aware Advantage Shaping for Deeper Reasoning (2026.findings-acl)

Challenge:	a new benchmark for evaluating the mathematical reasoning on large language models is being developed . popularity of reasoning benchmarks is leading to performance saturation and training set contamination.
Approach:	They introduce a benchmark for evaluating the mathematical reasoning on large language models . they find that models struggle with Mathador-LM, scoring lower than average 3rd graders .
Outcome:	The proposed benchmark improves performance on large language models . it also reduces test-set leakage into training data, a new study shows .

Copied to clipboard

Can Xie, Ruotong Pan, Xiangyu Wu, Zhang Yunfei, Jiayi Fu, Tingting Gao, Guorui Zhou

Placing Puzzle Pieces Where They Matter: A Question Augmentation Framework for Reinforcement Learning (2026.acl-long)

Challenge:	Reinforcement Learning with Verifiable Rewards (RLVR) has shown significant promise for enhancing the reasoning capabilities of large language models (LLMs).
Approach:	They propose a model-free method that refines credit assignment by leveraging the model's internal uncertainty signals.
Outcome:	Extensive experiments on five mathematical reasoning benchmarks show that the proposed method outperforms strong RLVR baselines on multiple model scales, including 1.5B and 7B.

Copied to clipboard

Yangyi Fang, Haolin Shi

Learning from Cognition: Enhancing RL Efficiency for LLM Reasoning via Hierarchical Metacognitive Decomposition and Refinement (2026.acl-long)

Challenge:	Reinforcement learning (RL) training on easy problems can cause overfitting and pass@k degradation, while training on hard problems yields sparse reward signals.
Approach:	They propose a hint injection framework that strategically identifies and provides critical reasoning steps during training.
Outcome:	Experiments on six mathematical reasoning benchmarks show that the proposed framework achieves comparable average performance to 32B baselines while preserving pass@k diversity across all k values.

Copied to clipboard

Zexu Sun, Yongcheng Zeng, Erxue Min, Heyang Gao, Bokai Ji, Dugang Liu, Xing Tang, Xiuqiang He, Xu Chen

The Emperor’s New Reasoning: Format Imitation Overshadows Genuine Mathematical Understanding in SFT (2025.emnlp-main)

Challenge:	Recent advances in Large Language Models have demonstrated notable inferential capacities via reinforcement learning (RL) however, “zero-RL” approaches relying on fixed prompt templates introduce substantial sampling inefficiencies for weak LLMs.
Approach:	They propose a hierarchical metacognitive RL framework that decomposes zero-accuracy problems into subproblems and prompts the policy to refine answers by referencing previous wrong solutions.
Outcome:	The proposed framework improves sample utilization and sample efficiency and accelerates convergence compared to baselines.

Copied to clipboard

Linyao Yang, Jian-Tao Huang, Yafei Lu, Zhenhui Jessie Li, Guirong Xue

Think Less, Know More: State-Aware Reasoning Compression with Knowledge Guidance for Efficient Reasoning (2026.findings-acl)

Challenge:	Recent advances in large language models (LLMs) have yielded impressive gains on mathematical reasoning benchmarks via supervised fine-tuning (SFT).
Approach:	They investigate the mechanisms behind SFT improvements in small-scale large language models by examining four key questions: (1) Are performance gains primarily due to format alignment rather than reasoning? (2) Can high-quality supervision encourage genuine reasoning? (4) Are format alignment gains consistent across model sizes and architectures?
Outcome:	The proposed models outperform the proprietary models on OlympiadBench and Omni-Math, but lack the brittleness of the models under perturbations to test their reasoning abilities.

Copied to clipboard

Yi Sui, Chaozhuo Li, Dawei Song

What If Consensus Lies? Selective-Complementary Reinforcement Learning at Test Time (2026.acl-long)

Challenge:	Existing CoT compression methods struggle to balance accuracy and efficiency . long CoT reasoning also introduces an overthinking phenomenon, authors say .
Approach:	They propose a framework that performs step-wise CoT compression by modeling stage-specific redundancy sources and integrating with a retrieval-augmented guidance.
Outcome:	The proposed framework reduces average response length by 59.9% while improving accuracy by 4.8 points over existing methods.

Copied to clipboard

Dong Yan, Jian Liang, Yanbo Wang, Shuo Lu, Ran He, Tieniu Tan

Robustness in NLP

Challenge:	Existing TTRL methods rely on positive pseudo-labeling strategies to enhance reasoning capabilities.
Approach:	They propose a test-time reinforcement learning framework that mitigates label noise amplification by deriving pseudo-rewards from majority voting consensus.
Outcome:	The proposed framework mitigates label noise amplification by implementing selective positive pseudo-labeling and entropy-gated negative p-labeled pruning.

CoVerRL: Breaking the Consensus Trap in Label-Free Reasoning via Generator-Verifier Co-Evolution (2026.acl-long)

Copied to clipboard

Teng Pan, Yuchen Yan, Zixuan Wang, Ruiqing Zhang, Guiyang Hou, Wenqi Zhang, Weiming Lu, Jun Xiao, Yongliang Shen

Token-Level Policy Optimization: Linking Group-Level Rewards to Token-Level Aggregation via sequence-level likelihood (2026.acl-long)

Challenge:	Label-free reinforcement learning enables large language models to improve reasoning capabilities . but as training maximizes self-consistency, output diversity collapses, authors say . authors propose a framework where a single model alternates between generator and verifier roles .
Approach:	They propose a framework where a model alternates between generator and verifier roles, bootstrapping each other.
Outcome:	Experiments show that CoVerRL outperforms label-free baselines on reasoning benchmarks . the framework can be used to improve reasoning abilities without ground-truth supervision .

Copied to clipboard

Xingyu Lin, Yilin Wen, Du Su, En Wang, Wenbin Liu, Zhonghou Lv, Jinchang Hou, Chenfu Bao

Good Reasoning Makes Good Demonstrations: Implicit Reasoning Quality Supervision via In-Context Reinforcement Learning (2026.findings-acl)

Challenge:	Group Relative Policy Optimization (GRPO) has significantly advanced the reasoning ability of large language models (LLMs).
Approach:	They propose a token-level framework that leverages sequence-level likelihood to link group-level rewards with individual tokens via token- level aggregation and introduces a KL-Divergence mask constraint that targets tokens with positive advantages and decreasing entropy to mitigate abrupt policy updates.
Outcome:	Experiments show that TEPO achieves state-of-the-art performance on mathematical reasoning benchmarks and reduces convergence time by 50% compared with GRPO/DAPO.

Copied to clipboard

Tiehua Mei, Minxuan Lv, Leiyu Pan, Zhenpeng Su, Hongru Hou, Hengrui Chen, Ao Xu, Deqing Yang

What Makes a Good Curriculum? Disentangling the Effects of Data Ordering on LLM Mathematical Reasoning (2026.acl-long)

Challenge:	Reinforcement Learning with Verifiable Rewards (RLVR) improves reasoning in large language models but treats all correct solutions equally, potentially reinforcing flawed traces that arrive at correct answers by chance.
Approach:	They propose a method that reweights rewards by a factor approximately proportional to Evidence Gain and assigns higher weights to high-quality traces without requiring costly computation.
Outcome:	Experiments on mathematical reasoning benchmarks show that Reinforcement Learning with Verifiable Rewards (RLVR) improves reasoning in large language models but treats all correct solutions equally.

Copied to clipboard

Yaning Jia, Chunhui Zhang, Xingjian Diao, Xiangchi Yuan, Zhongyu Ouyang, Chiyu Ma, Soroush Vosoughi

Pru-CoT: Towards Efficient Reasoning Distillation via Pruning Chain-of-Thought (2026.findings-acl)

Challenge:	Curriculum learning (CL) orders data corpus by difficulty, but prior work employs disparate difficulty metrics and training setups.
Approach:	They propose a framework that decomposes curriculum difficulty into five dimensions: Problem Difficulty, Model Surprisal, Confidence Margin, Predictive Uncertainty and Decision Variability.
Outcome:	The proposed framework decomposes curriculum difficulty into five dimensions . the results show that no curriculum strategy dominates universally .

Copied to clipboard

Han Liu, Shuotian Ma, Hui Li, Xiaotong Zhang, Fenglong Ma, Hong Yu

MARD: Module-Aware Reasoning Distillation for Language Models with Adaptive Supervision (2026.acl-long)

Challenge:	Existing heuristics fail to capture global causal logic due to rigid rules and limited search spaces.
Approach:	They propose a framework that extracts the essential logical structure from reasoning chains.
Outcome:	Experiments show that Pru-CoT models generate more compact reasoning paths compared to models trained on verbose data.

Copied to clipboard

Wenqi Yang, Jianjun Li, Zhibo Zhang, Mingqian Ding, Yushen Fang

CE-GPPO: Coordinating Entropy via Gradient-Preserving Clipping Policy Optimization in Reinforcement Learning (2026.acl-long)

Challenge:	Multi-step reasoning remains challenging for language models with limited capacity . et al., 2025) demonstrate remarkable reasoning capabilities across diverse tasks .
Approach:	They propose a module-aware reasoning distillation framework that explicitly targets key Transformer components for effective reasoning transfer.
Outcome:	The proposed framework targets key components for effective reasoning transfer . it adopts an offline distillation setting, where a strong teacher model provides reasoning trajectories in advance .

Copied to clipboard

Zhenpeng Su, Leiyu Pan, Minxuan Lv, Yuntao Li, Wenping Hu, Fuzheng Zhang, Kun Gai, Guorui Zhou

Dipper: Diversity in Prompts for Producing Large Language Model Ensembles in Reasoning Tasks (2025.emnlp-main)

Challenge:	Existing methods for proximal policy optimization discard valuable gradient signals from low-probability tokens due to the clipping mechanism.
Approach:	They propose an algorithm that reintroduces gradients from clipped tokens in native PPO in a gentle and bounded manner.
Outcome:	The proposed algorithm outperforms strong baselines on reasoning benchmarks on different model scales.

Copied to clipboard

Wenyang Hu, Gregory Kang Ruey Lau, Liu Diwen, Chen Jizhuo, See-Kiong Ng, Bryan Kian Hsiang Low

Beyond Experience Retrieval: Learning to Generate Utility-Optimized Structured Experience for Frozen LLMs (2026.acl-long)

Challenge:	Existing methods for large language models rely on sequential queries . however, existing methods rely heavily on sequential querying .
Approach:	They propose a training-free framework that transforms a single LLM into an effective inference-time ensemble.
Outcome:	The proposed framework outperforms existing models on reasoning benchmarks, such as MATH, and improves on a DIPPER ensemble of three Qwen2-MATH-1.5B instances.

Copied to clipboard

Xuancheng Li, Haitao Li, Yujia Zhou, Yiqun Liu, Qingyao Ai

Information Retrieval

Challenge:	Large language models (LLMs) are largely static and often redo reasoning or repeat mistakes. Prior experience reuse relies on external retrieval, which is similarity-based, can introduce noise, and adds latency.
Approach:	They propose a lightweight plug-in that stores experience in its parameters and generates a structured, instance-tailored experience entry in a single forward pass to guide a frozen LLM executor.
Outcome:	Experiments on mathematical reasoning benchmarks show consistent accuracy gains across executors with low overhead.

Orchestrating Tokens and Sequences: Dynamic Hybrid Policy Optimization for RLVR (2026.findings-acl)

Copied to clipboard

Zijun Min, Bingshuai Liu, Ante Wang, Long Zhang, Anxiang Zeng, Haibo Zhang, Jinsong Su

Rhombus: Incentivizing Coordination in Parallel Thinking through Reinforcement Learning (2026.findings-acl)

Challenge:	Existing RLVR algorithms focus on different granularities and have complementary strengths and limitations.
Approach:	They propose a framework for reinforcement learning with verifiable rewards that bridges RLVR and GSPO . group-level importance ratios are used to update a policy, which preserves fine-grained credit assignment .
Outcome:	The proposed framework outperforms existing methods on seven reasoning benchmarks.

Copied to clipboard

Ziyuan Nan, Qi Yi, Di Huang, Yutong Wu, Guanhua Huang, Xue Gong, Kejiao Li, Yuhao Jiang, Chenchen Zhang, Zenan Xu, Xing Hu, Bo Zhou

The Confidence Paradox: Unveiling the Latent Discriminative Power of Diffusion Large Language Models in Mathematical Reasoning (2026.findings-acl)

Challenge:	Parallel thinking is a promising avenue for scaling test-time compute in Large Language Models . however, coordinating the exploration and aggregation stages remains challenging .
Approach:	They propose a parallel thinking framework that explicitly incentivizes coordination between components via end-to-end reinforcement learning.
Outcome:	The proposed framework improves accuracy by 6.0% over long chain-of-thought baselines while reducing wall-clock latency by 39.4% under matched token budgets.

Copied to clipboard

Yansi Li, Gongshen Liu, Zhuosheng Zhang

Reinforced Efficient Reasoning via Semantically Diverse Exploration (2026.acl-long)

Challenge:	Diffusion large language models (DLLMs) are a promising alternative to autoregressive (AR) generation, offering token-level probabilities under bidirectional context.
Approach:	They propose to use diffusion large language models to generate token-level probabilities under bidirectional context and to examine the calibration paradox inherent to their native uncertainty estimates.
Outcome:	The proposed model outperforms AR baselines on mathematical reasoning benchmarks and is highly miscalibrated on reasoning benchmark.

Copied to clipboard

Ziqi Zhao, Zhaochun Ren, Jiahong Zou, Liu Yang, Zhiwei Xu, Xuri Ge, Zhumin Chen, Xinyu Ma, Daiting Shi, Shuaiqiang Wang, Dawei Yin, Xin Xin